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ABSTRACT 

~" ' ~ "" t Three* factors tend to remove classroom teachers from 

any direct benefits from formal research: %1) %hm difficulty of 
replicating the exact circumstances of the study in the classroom; 
(2-). the scarcity- of research which claims to show ^a "significant 

, diff irenca B ; and* (3) the complexity of reporting, in formal research 
journals and reports, which tends to intimidate individuals who lack 
needed interpretation skills* ^his paper examines these problems and 
proposes an approach to dealing with the problems of gene r al i zabi 1 i t y 
of research findings and accessibility of research evidence. It is " 
suggested that laboratory schools are appropriate s ites under which 
to undertake research that can obtain causal evidence of the success 
or failure of new hypotheses which are directly applicable to the 

. real circumstances of the classroom, It is also suggested that 

\ colleges of education maintain a constant emphasis upon the value of 
\ the teacher's remaining in touch with current research. ( JD) r * 
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Educational research is of most value, to the practitioner when it 
offers a clear hope for some" improvement of learning in students. , 
However* there are at* least three factors which^tend to ferno-ve class- 
room teachers from any direct benefits from formal research. First is 
the scarcity of ^research which claims to show -a "significant differ- 
ence ; 11 second , the di fficuHty .of replication of the .exact circumstances 
of the study in our classrooms, and, final ly, the complexity of report- 
ing of formal research journals and* reports whfch tends to Intimidate 
most educational practitioners who may lack the needed skills of inter- 
pretation, x - 

This paper examines these problems and proposes an approach 'to deal- 
ing with the problems of external validity ( g e n e r a 1 i z a b i 1 i ty ) and ac- 
cess ibi 11 ty^Df research evidence. The point of view expressed in this 
paper grew out of wide acquaintance with current practice in teaching 
in public schools, the frustration of trying to find an organized body ' - 
of research concerning a single educational problem,, and, finally, the 
conviction that laboratory schools might offer a unique source foi^ at- 
tacking the problems described, . . 

Much educational research is concerned with causality - that is » what s 
"causes 11 learning to occur, whether It be acquisition of facts, under-' 
standing of concepts, development of attitudes and 'behaviors , etc. 
With this research it is usually desirable to be able to generalize 
the results of experimentation concerning causal relations to a larg er 
population or to a ^similar ^population at a later time. If a .curricular 
innovation can be shown to affect student learning positively in a given 
setting, one is always hopeful that it can produce a similar result in 
other settings/ _ > ^ ' ' 



Ennis * (3; 6) suggests that causal generalisations may be justified * 
— ' 4 7------ - = , , * J . # ->* 

by* "derivation" or by "warranted induction* 11 That is, that we can "gen- 
eralize from one or more specific causal statements, on the ground that 
the subjects of these statements are appropriately typical of the group 
covered by the generalization,* 1 This kind of generalization if based 
on what Campbell and Stanley call "external validity" and it has been 
the subject of much concern and discussion by educational experimenters . 
By examining the pursuit of externa*! validity according rto the points 
% of view of Campbell and Stanley* Bracht and Glass^jitfd others **this 

paper will attempt to synthesize these contrasting -but similar; 4 opinions 1 
into an approach that seems to havf some merit. #* 

The question to be examined might be stated* M Can specific causal 
statements of *the type 'x caused be expanded into general causal 
claims, that is* 'xjs cause Jf's?" And,, obviously we are then concerned 

with the corollary question, 11 If so* how?'' 

* ■ . . ■ * * 

* * * 

tt would seem :o be an easier tas*: to take the opposite point of view 
and say that general causal statements can never be confidently arrived 
at by this method. An explanation of the singular rausal elainf tha£ 
"x caused which means* M x_ occurred and* in the ci rcunfstances s £ would 
not haye occurred if- £ had not* 11 can be used to easily refute this point 
of genera iizability. One need only insist that "in the circumstances" m - 
be applied in such a strict way that such circumstances can never be 

exactly duplicated again, Even scientists* with much more control of 

their circumstances than educationalists* maintain that their laws apply 
only to all presently or previously known situations. Future discover- 
ies may make current laws useless just as discoveries in the past have 
made inapplicable laws previously thought to be true. Certairtly If we 

\ ' - 



thi nk abou t our own experience , we recognize not onl y ^w ide diversity 
of individuals within our own classes this year,, but collective differ- 
ences in these groups from year, to ye_y% These differences are pre sum-* 
ably based on some rough estimate of both cVass means and variability. 
And so, even if we believe that treatment £ caused learning £ to occur 
thi s time/ what confidence can we have that it will happen again? If 
we turn on a. faucet* we. have every expectation that water will come out, 
that i s , that" turning on faucets causes woter to come out. This is true 
even though most people have at some time had^the experience of turning 
on a faucfet and not getting _my water . erhaps the -water jw^s* turned off 
elsewhere or a pipe was broken or fewn v rozen t but still we assume that 
conditions will normally be such that water wi 1 1 -come forth. However, 
because of the variability of educational settings and human subjects 
we are 'reluctant to make the same kinds of assumptions about replication 
.of results from similar treatments* 

Both Campbell and Stanley and Bracht and' Glass have established some 
extremely confining /requirements for external - validity . Campbell and 
Stanley refer to Hume's truism that "generalization Is never fully jus- 
tified logically, "(2: 17) They agree, logically, but continue that we 
do make some quallffed guesses ant? estimates based on empirical obser- 
vations of our field and the results of previous attempts at general i* 
zafion. They insist, however, that we employ careful control over a 

number of variables. Two that will be considered here are control *of 

■ = ■ — — _. . . — _ ^..__^ = _ = __ : — r _____ ______ „ _ . _ _ , , — _ — ___ — __ — _ — ___ _ _ 

sampling and experimental control of environment. 

Campbell and Stanley are quite concerned with random sampling" in 
order to insure the representativeness of the experimental sample to 



the population. Bracht. and Glass, on the other hand, are willing to 
allow generalize:?- j ev* r without random selection, to a larger popula- 
tion "like tV-- n a;. v f -quoting Corpfield and Tukey), (1: 441) \., 
He use i, -or. iures to try to establish that samples selec- 

IS - 

tea for f y . # w 3 ti ve of ^o abnormal population and therefore 

.results :-• apipi ir :hat population. 1 However, in education, the 
"subjec i* n« s n •■ ^.^n n intact classroom rather than an individual' 
student j K.„. "in, classroom groups (samples) are 'truly represen- 
tative Qi t; na: . rms 1 population? When grouped together totally, stu- 
dents ar* n; h Miht "normally distributed 1 ' in most respects, but we do. 
not teflfci , .i entire populations, but rather small samples/ If such care* 
must be taken in selection pf a sample In order* to guarantee its repre- 
sentativeness for validity of selection, must not equal care be taken 
to find a similar "group of learners (classroom), to which we can apply 
the results of our research? It is doubtful that very many Intact" class- 
rooms fit the "normal M population of students, and so would not be gpp-ro- 
priate groups to whom w^ might safely apply the generalization from our 
carefully selected, sample. 

This fact of educational life (if it is, indeed, a fact) would seem 
to call for more research directed toward individuals or specified groups 
of learners, With either stratified, sampl ing or some analysis of treat- 
ment effects on Individual learners we could, perhaps, distinguish be- 
tween results with boys and girls, high and Tow-I.Q. or aptitude, or 

different^degrees-of student-moti vati on Whether— the _e vi dence conc erns 

teaching, materials, process or method, learning modes, - teacher/class 
interaction, or other such factors, its effect can be studied and, where 



obfripusjy good or bad resul ts ^oceurv noted/ * 

I 

Such a focus on subsets^of the total population would, also tend -to 
lessen another major deterrent to the applicability of educational re- 
search, That is , the failure to obtain a finding of statistically, sig- 
nif leant difference. Unless the measure being evaluated is extremely 
powerful or given a very long chance to act. It is .unlikely to affect the 
total' classroom in such a fashion as to significantly alter measures of * ? 
central tendency* There would seem to be a much 'greater likelihood of 
finding an effect on some learners, and Identification and description 
of that group could provS^ equally valuable, 

With respect to the second point of. control of experimental conditions 
Bracht and Glass suggest that it may be more Important to control for thi 
than to ^control for sample differences (1: 452).* Way agree that situa- 
tions pfobably vary more than individuals from one setting to the next. 
This point seems valid and yet one ^f .the principal variables in an edu- 

# M 

■ . e - 

cational setting must be^ the experimenter/teacher and that can not be 
controlled' in other situations. And, further, in educational research, 
becauSe of the importance of interaction among events and* ^laments of 
events , we are unable to clearly separate causes from conditions or par- 
tial causes from the main cause, These difficulties would demand care- . 
ful description of all aspects of the setting - = explicit description 
of the independent variables (1: 455) . Included must be the length of 
treatment, possibility of novelty or disruptive effects, learning atmos- 
phere of the classroom, teacher) s experience with the Independent vari - 
able, and others, * 

Based on the' above considerations the following proposal is off erred 



as a means of approaching causal evidence - experimental or otherwise -, 

■ ■ ■ _ * i rf , 

that might be of some possible value to the educational practitioner, \ 

In education we have either accepted every new idea which has recei ved\ 
any kjnd of promotion - with -or without evidence of its value in effect- 
ing desirable change - or we have Ignored research because it has not - 
"proved" that x can cause j/ to occur or because x has been so narrowly \ 
defined as to seem* inapplicable in our situations perhaps we rni-ght *agree 
to accept research evidence (whether. by carefully controlled experimenta- 
tion or simply observation of what 'works*) as tentative support for a, 
hypothesis and *t ry out; the hypothesis ourselves, not uncritically accept- 
ing it as truth, but always looking for new evidence to either confirm 
or deny its applicability in this. new situation and possibly to modify 
its claims tn the process, f 

We should allow far a more flexible interpretation of "normal condi- 
tions" or "in the circumstances" to encourage further use o*f research 
evidence. * Controls must certainly be as strict as possible in. order to * 
describe what actually happened, but the fodut might shift more to*' a 
leading of the study to give the best ^chance for finding significance, 
e.g . can this treatment bring about a desired change under some condi- 
tions or to some of our population. With the use of more complex fac- 
tori a! designs and multiple analysis of variance becoming more common 
and accessible in educational research (even for the non-sophisticated 
researcher) it is possible to determine fflore clearly the Interactions 
referred to above *and to Identify the parts of the population; most 
a f f e eted by - the t reatmen t * — Re^li ca.ti.on_. mus t bee^ - even re pi id- 

eation in an action setting (classroom) where less control can be \ 



achieved - ancUreported for* Whatever valu'e it ijiay be worth. As a small 

* body of. such evidence Is amassed, it may 'become clear thBt the stated 

hypotheses is being supported in every instance. Or, further repli^ * 

- # 
P tipn may tend to dispute the original finding* giving cause to' suspect 

/ 

the validity of that hypothesis, It should- be apparent that negative * 

findings are equally important as positive support. Seldom does one 

read a report of an educational project ^or proposed solution to a prob- 

lern that failed. Much* could be. learned from-sueh reporting, if condition 

that led to the failure can be , identified /and described. ' 
* ** 
In order for such an apprqach to causal research to succeed, at least 

three conditions musl^be present. First, someone must be willing to gatl 
the evidence; that Is, to think creatively about an educational problem, 
to try out new Ideas in practice, to keip* ateurate records of the events 
and to write up the results," Secondly, there must be a means of dis- 
seminating the information which has been accumulated* Someone must 
j . 

collect-and publish, the .tentative evidence which has been gathered in 
order that It become available for further, study and/or use. _ Finally, 
these reports must get into .the hands of the classroom teacher in such 

a .form that the^can (1) be understood by novices in research and (2) 

* 

be perceived by the teacher as having some practical value, that Is, 
meeting a Vecogrii zed need of that teacher* 

It remains for a plan to be developed which will bring Into actuality 
each of the three conditions jost described. This paper will only sug- 
gest some possible beginnings, with special reference to the two, groups 
represented in* this meeting: the National Association of laboratory 
Schoe^- a nd -the_Amerlca n As soci a t i o n of Colleges, for Teacher Education. 



%- * first, one must identify ^an initiator for tte gathering of evidence. 
It may well be that laboratory schools offer the single best repository 
of resources to undertake this task. In 1982, one- of the goals*. identi- 
fied for helping ensure th$ survival of laboratory schools- in the igSO's 
was* to become more involved in research. Faculty in laboratory schools 
generally are interested, in educational happenings beyond their 'class- 
rooms and, additional ly , have access to the resources of a university's 
research facilities. Laboratory schools also contain students who re pre 
sent, col'lecti vely , all geographic, economic, social /cultural , ,and Intel 
lectua-1 levels. This should permit the same range *of stratification 
in sampling as a public school population and so -make the rtesults of our 
studies more clearly genefal izable'* For these reaspns it is proposed ^ 

that laboratory schools become actively involved in initiating a planned 

* • ' * * * 
and organized program of "evidencf, gathering" (research, i-f you will) 

concerning some well- defined educational concerns. - * 

Witfr respect to the second necessary condition - a channel for tommun 

i cation the NALS has- available both the journal and the Newsletter 

as vehicles for internal reporting of, evidence- Indeed, the Journal 

currently features articles describing "activities, units, and programs 

that are considered by individual* teachers to be of distinct -educational 

value, it remains only for some expansion and focus to be given to 

r these efforts in order to' adequately Inform the NALS membership* This 

doe$ not, however* address the problem of cdmmuni cation with'' t^ie general 

teaching population. Whether the needed pubUi cation resources to accom- 

plish this taik are now in existence or need to be created is a subject \ 

which this proposal can not address/ ^ 

perhaps the most difficult* and controversial condition^ tills pro- ^ 

posal is the last: creating an interest snd a willingness on the part , 



of classroom teachers to become involved in these research efforts. 
-* What can be done to Stimulate interest in t t he?e ongoing attempts at fur- 
thering knowledge -about, our educational situation*, even given access to 
clearly described reports of current research efforts? ' Wi th_ the many 
tasks and problems .facing^ classroom teachers 'on a dpi ly 'basis •.' 1s there 
W hope that they can find the time or interest for such esoteric matters 
as conducting or even taking note of research activities? At thfs point 
the ^Colleges of Education wduld se§m to have^a crucial role to .play. 

In both pre -service (undergraduate) and -in-service (graduate) education 

* . # * 

the student must be challenged and encouraged to see the merits of con tin- 

ual involvement in educational research, whether as a practicing researcher 

or merely a "consumer" of research. This does not call for more research 

courses, but rather a constant emphasis on the value to "the educator Of * 

remaining in touch with current research, both formal and informal, 

o . - " • . ~ : ■ ' j- . ■ 

There is a second jpeans by which teacher eduction colleges might * 

become more of t a force in productive research . For those schools with . 

, * . - I ■ - ■■ 

large graduate programs, at either the masters or doctoral level, it 

might* be .desirable to- identify certain areas (questions) for invfestl- 

"gation by_ student researchers . This approach has proven successful in 

some 'Schools. Graduates? 1 , students could be encouraged to choose research - 

topics which were directly related to thpse guestiohs or ip replicate 

previous studies in the area of investigation,, 5 This. procedure might ' 

produce a more worthwhile body of research <evi dence. than the current . 

"scattergun" jnethod,of selection of topics** 1 0 , - - 

F.inalTy,.how can, educators, then , decide wKen* to accept new, ideas 
. .-.•*.«** 
^ • i > 

or suggested causal relationships for the*i r own use without fall trig 

-into 0 one of. the extreme^ppsitions referred to earlier ( 'bandwagon .ac^ * \ * 

> ■■* - * ■ ■ . " 1 ■ * * ". \ ' 

#• * . . - . • ' . • *;.*:. . • ■ v . 

" : ' ■ ' ' ' - ■ , . • . v ••'•»••" ■ 

- . . ■ * - .. ■ • - * " , \ . * * 
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ceptance or demand for total 'proof 1 )? This is, perhaps, the 41101 1 basic 

* ' - , r * * = 

question for. the position presented here of greater emphasis on action 
research. The only answer to be of f erred is admittedly vague*. To set 
up some precise guidelines fo'r either the population to whom the "results ' 
may reasonably be applied or the degree of success, needed to make repli- = 

m i . , • " • * . * " 

cation worthwhile. only moves us further away 'from the original thesis, 
' And .soV the only answer ^suggested at the present time is that 'descriptions. 
t>f "the observed cause a?jd effect relationships (methods and results) 

. r '■ \ - _ _ 

should- be measured and reported as carefully and precisely as possible - 
especially 'with^, regard to pertinent characteristics of the sbmple.,and* 
exfict prOGedures of the treatment .- and then the responsibility for de~ 
elding on further use rests" with the individual reader of the results* 
If our own learning situation is" Similar 1 in relevant respects, and if 
we feel that the reported 'success 1 is significant, then "we. may make the 

decision to try our own, replication and use , our results to further con- 

* . / . 

firm, modify," or refute the original hypothesis, 
V These statements ^should* in no way be Interpreted as a defense of sloppy 

* ' / . .7 . - .. . . > 

research or illogical thought about caudal relations. There still Is a 
need for an emphasis on pure research/ and th^ best and most carefully 
controlled ^work 'where it can' be accomplished., „ f But in education, the 

greater ^problems seem to be finding/ research that claims to make a dif- 

*■ = 

fere nee and putting Into practice the results of that research. Even 
the. 1 i mi ted ''evidence wfe have' discovered seems to be mainly left on the 
31 binary shelves or, perhaps, cJi scps/sad toy .graduate students -in research 



courses- , "-•'-/ 
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It is hoped that the ideas expressed in this paper will lead to some 
thought and discussion which might eventually bring about a closer inte 
gration of educational research and practice. 
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